The present invention pertains to a screen image observing device having a head tracking function, and more particularly, to a screen image observing device such as a head mounted display (HMD) which provides the sensations of virtual reality to the observer.
Screen image observing devices having a head tracking function are conventionally known as being those devices which display appropriate graphical computer screen images in response to movement of the observer""s head. These devices also typically have a remote control system in which a camera is remotely controlled based on head tracking signals. Screen images, captured by the camera, are observed through an HMD. By utilizing a screen image observing device with a head tracking function in this way, it is possible to display screen images which have increasingly realistic appeal.
In conventional screen image observing devices, however, a discrepancy exists between the movement of the observer""s head and the displayed screen image. A certain time interval is required for image processing after a head movement is detected but before a new image, based upon the result of the head movement, is displayed. When the observer""s head is essentially stationary, the discrepancy cannot be perceived to any significant extent, even if movement of the head does not match the displayed screen image. However, when the observer""s head moves quickly, the discrepancy can be perceived. Such a discrepancy between visual information and physically perceived acceleration causes mental discomfort and can cause the observer to become nauseous, due to his vestibular sense. The faster the observer""s head moves, the larger the discrepancy in the displayed screen image, and the more severe the discomfort and nausea.
An object of the present invention is to provide a screen image observing device and method that reduces, to the extent possible, the discrepancy that is perceived by the observer between the movement of the observer""s head and the displayed screen image.
In order to attain the object described above, a screen image observing device is provided having a head tracking means, which detects the movement of the observer""s head, and a screen image generating means, which generates image data based upon the result of the detection performed by the head tracking means and supplies the image data to a display means. The screen image observing device also has a control means for controlling the screen image generating interval, used by the screen image generating means, in response to the detection performed by the head tracking means.
In this embodiment, the movement of the head is first detected by the head tracking means. The screen image generating means generates screen image data in accordance with the detection result obtained by the head tracking means, i.e., it generates screen image data to keep up with the head movement. The control means controls the screen image generating interval used by the screen image generating means. When the screen image generating interval is accelerated by the control means, the time period after a head tracking signal is detected but before a screen image based on the detection result is displayed, is reduced. Therefore, the discrepancy between the displayed screen image and the view in the direction in which the head faces is reduced accordingly.
Head movement encompasses all movements of the head, including rotational movement, back and forth movement and side to side movement. It is therefore possible to calculate the direction in which the head faces in these dimensions, in accordance with the detection result obtained by the head tracking means. In this case, the screen image generating means executes a process to generate, from the original data, screen image data having parallax information that corresponds to the right and left eyes, respectively. The control means will control the screen image generating interval for this screen image data.
In another aspect of the present invention, a screen image observing device which has an image capture means, and the screen image generating means generates the screen image data using the captured image data from the image capture means.
In this embodiment, an image of an object in the direction corresponding to the detection result, obtained by the head tracking means, is captured by the image capture means. The screen image generating means generates screen image data using this captured image data. The control means controls the screen image generating interval used by the screen image generating means.
In a further aspect of the present invention, the control means reduces the screen image generating interval, used by the screen image generating means, to below a prescribed time period according to the formula;             V      ⁢              xe2x80x83            ⁢      θ        =                  2        xc3x97        Y            X        ,
wherein:
Vxcex8 is the angular speed, in degrees per second, of the rotational movement of the head which is calculated from the result of the detection performed by the head tracking means;
X is the number of pixels in the image, displayed by the display means, corresponding to a one-degree angle of view on the retina; and
is the frame rate in frames per second.
A larger angular speed of the rotational movement of the head results in a larger perceived discrepancy between the view in the direction in which the head faces and the displayed screen image. Therefore, in this embodiment, the control means determines whether or not to reduce the screen image generating interval, used by the screen image generating means, in accordance with the angular speed of the rotational movement of the head. Using this control means, where the head moves only slightly and the observer cannot perceive the discrepancy, the control means does not shorten the screen image generating interval. On the other hand, where the angular speed of the rotational movement of the head equals or exceeds a certain level, the observer can perceive the discrepancy, and, therefore, the screen image generating interval is reduced so that the discrepancy between the displayed screen image and the view in the direction in which the head faces may be reduced. The constant value 2xc3x97Y/X indicates the angular speed of the head at which the displayed screen image deviates from the image actually observed by two pixels during one frame display period.
In yet a further aspect of the present invention control means determines whether or not to obtain a second head tracking signal in response to a first head tracking signal received from the head tracking means, and where it is determined that a second head tracking signal is to be obtained, reduces the screen image generating interval, used by the screen image generating means, to below a prescribed time period.
The control means normally detects a first head tracking signal and performs the ensuing control based on this signal, and a screen image is displayed at a prescribed display time. Here, if the amount by which the screen image generating interval should be reduced is estimated in accordance with the first head tracking signal, and a second head tracking signal, generated after a smaller time lag than used for the first head tracking signal, is detected, subsequent control is performed based on this second head tracking signal. A screen image is displayed at a prescribed display time and control is performed based on subsequent head tracking information, resulting in a reduced discrepancy between the displayed screen image and the view in the direction in which the head faces. Such control is increasingly effective as the head moves faster.
In a further aspect of the present invention, the control means controls the screen image generating interval by controlling the number of colors per pixel in the screen image data.
In this embodiment, the data amount is reduced by reducing the number of colors per pixel in the screen image data, for example, and the screen image generating interval may be reduced to that extent. Since the motion tracking capacity of the human eyeball is said to be 35 degrees per second, when the speed of the head movement exceeds a certain level, focusing is not fixed or functioning (i.e., even though the observer sees an object, he is not fully perceiving its colors or edges). Therefore, even if the number of colors is reduced, the observer does not feel discomfort.
In another aspect of the present invention, the control means controls the screen image generating interval by controlling the number of pixels for one item of screen image data.
In this embodiment, a screen image for one frame may be expressed using less screen image data by increasing the number of pixels corresponding to one item of screen image data, for example. When the screen image data for one frame is reproduced from the original data stored in memory, the reproduction time is reduced by thinning the pixel data. In addition, because the data amount is reduced, the transmission interval also is reduced when the data is transmitted. In other words, the screen image generating interval is reduced. Therefore, as with the previous aspect where the angular speed of the rotational movement of the head exceeds a certain level, the observer can observe the image without feeling discomfort.
In yet another aspect of the invention, the control means controls the processing means in response to the result of the detection performed by the head tracking means, and the processing means transmits the captured image data to the screen image generating means after controlling the data amount based on the signals received from the control means.
In this embodiment, data captured by means of the image capture means is transmitted to the screen image generating means after being controlled in terms of its data amount. Therefore, the amount of data transmitted from the image capture device to the screen image generating means may be reduced. Where the data amount to transmit is small, the time interval required to complete the transmission of the data is reduced. In addition, by reducing the amount of data transmitted from the image capture device to the screen image generating means, wireless transmission becomes possible, allowing the device to become a screen image observing device having a remotely controllable image capture device.